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Critical Elements Checklist: Extracted from Standards and Assessment Peer Review Guidance, 4/28/04 



Purpose and Uses: This checklist was designed for use by a state team to conduct a preliminary 
self-analysis of the state’s system of standards and assessments. In using this checklist, it is 
essential that reference be made to the full document: Standards and Assessments Peer Review 
Guidance: Information and Examples for Meeting the Requirements ofNCLB , Office of 
Elementary and Secondary Education, April 28, 2004 (available in both Word and PDF at 
http://www.ed.gov/admins/lead/account/saa.html) . 



SECTIO] 


N 1: CONTENT STANDARDS 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






1.1 

(a) Has the State formally approved/adopted, by May 2003, challenging academic 
content standards in reading/language arts and mathematics that - 

• cover each of grades 3-8 and the 10-12 grade range, or 

• if the academic content standards relate to grade ranges, include specific content 
expectations for each grade level? 

AND 

(b) Are these academic content standards applied to all public schools and students in 
the State? 






1.2 

Has the State formally approved/adopted, academic content standards in science for 
elementary (grades 3-5), middle (grades 6-9), and high school (grades 10-12)? This 
must be completed by school year 2005-2006. 






1.3 

Are these academic content standards challenging? Do they contain coherent and 
rigorous content and encourage the teaching of advanced skills? 






1.4 

Did the State involve education stakeholders in the development of its academic 
content standards? 
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SECTIO] 


N 2: ACA] 


DEMIC ACHIEVEMENT STANDARDS 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






2.1 

Has the State formally approved/adopted challenging academic achievement standard 
in reading/language arts and mathematics for each of grades 3 through 8 and for the 
10-12 grade range? These must be completed by school year 2005-2006. 

Has the State also approved/adopted alternate academic achievement standard for 
students with the most significant cognitive disabilities in reading/language arts and 
mathematics for each of the grades 3 through 8 and for the 10-12 grade span? If 
alternate achievement standards have not been developed and approved, then the 
alternate assessments for students with disabilities must be based on grade-level 
achievement standards. 

For students under section 602(3) of the Individuals with Disabilities Education Act 
with the most significant cognitive disabilities who take an alternate assessment, a 
State may, through a documented and validated standards-setting process, define 
alternate academic achievement standards, provided those standards (1) are aligned 
with the State’s academic content standards; (2) promote access to the general 
curriculum; and (3) reflect professional judgment of the highest achievement standards 
possible. 






2.2 

Has the State formally approved/adopted academic achievement descriptors in science 
for each of the grade spans 3-5, 6-9, and 10-12 (not due until school year 2005-2006)? 

Has the State formally approved/adopted academic achievement standards cut scores 
in science for each of the grade spans 3-5, 6-9, and 10-12 (not due until school year 
2007-08)? 

Has the State also approved alternate academic achievement standards for students 
with the most significant cognitive disabilities in science for each of the grade spans 
tested? If alternate achievement standards have not been developed and approved, then 
the alternate assessments for students with disabilities must be based on grade-level 
achievement standards. 






2.3 

1. Do these academic achievement standards (and alternate achievement standards, if 
applicable) include for each content area - 

(a) At least three levels of achievement, including two levels of high achievement 
(proficient and advanced) that determine how well students are mastering a 
State’s academic content standards and a third level of achievement (basic) to 
provide information about the progress of lower-achieving students toward 
mastering the proficient and advanced levels of achievement; and 

(b) descriptions of the competencies associated with each achievement level; and 

(c) assessment scores (“cut scores”) that differentiate among the achievement levels 
and a rationale and procedure used to determine each achievement level? 

2. And, if the State has adopted alternate academic achievement standards, has it 
documented that it has — 

(a) implemented guidelines for IEP teams to apply in deciding when an individual 
student should be assessed on the basis of alternate achievement standards; 
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SECTIO] 


N 2: ACA] 


DEMIC ACHIEVEMENT STANDARDS 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






(b) ensured that parents are informed when a child's achievement will be based on 
alternate achievement standards and any possible consequences imposed by the 
LEA or State; 

(c) reported separately the number and percent of those students with disabilities 
assessed against alternate achievement standards, those assessed on an alternate 
assessment against grade-level standards, and those included in the regular 
assessment (including those administered with appropriate accommodations); 

(d) documented that students with the most significant cognitive disabilities are 
included in the general curriculum to the extent possible; 

(e) taken steps to promote use of appropriate accommodations for students tested 
against grade-level standards; and 

(f) provided information for teachers and other staff regarding appropriate test 
administration practices, including use of accommodations for students with the 
most significant cognitive disabilities? 






2.4 

Are the academic achievement standards (including alternate achievement standards as 
applicable) applied to all public elementary and secondary schools and all public 
school students in the State? 






2.5 

How has the State ensured alignment between challenging academic content standards 
and the academic achievement standards? 

If the State has adopted alternate achievement standards, how has the State ensured 
alignment between its academic content standards and the alternate academic 
achievement standards? 






2.6 

How did the State document involvement of diverse stakeholders in the development 
of its academic achievement standards and (if applicable) its alternate achievement 
standards? 
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SECTION 3: OVERVIEW OF THE STATEWIDE ASSESSMENT SYSTEM 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






3.1. 

In the chart below indicate your State’s current assessment system in reading/ 
language arts and mathematics in grades 3 through 8 and for the 10-12 grade range 
using the abbreviations to show what type of assessments the State’s assessment 
system is composed of: (a) criterion-referenced assessments (CRT); or (b) 
augmented norm-referenced assessments (ANRT) (augmented as necessary to 
measure accurately the depth and breadth of the State’s academic content standards 
and yield criterion-referenced scores); or (c) a combination of both across grade 
levels and/or content areas. Also indicate your current assessment system in science 
is aligned with the State’s challenging academic content and achievement standards 
at least once in each of the grade spans 3-5, 6-9, and 10-12. A State may have 
assessments in reading or language arts depending on the alignment to the State’s 
content standards; both are not required. Please indicate, using the abbreviations 
shown, the grades and subject areas with availability of native language assessment 
(NLA) or various alternate assessments (AA-SWD for an alternate assessment for 
students with disabilities based on grade-level standards; AA-LEP for an alternate 
assessment for students with limited English proficiency based on grade-level 
standards, and/or AA-AAS for an alternate assessment for students with the most 
significant cognitive disabilities based on alternate achievement standards). 






3.2 

If the State’s assessment system includes assessments developed or adopted at both 

the local and State level, how has the State ensured that these local assessments meet 

the same technical requirements as the statewide assessments? 

(a) How has the State ensured that all local assessments are aligned with the State’s 
academic content and achievement standards? 

(b) How has the State ensured that all local assessments are equivalent to one 
another in terms of content coverage, difficulty, and quality? 

(c) How has the State ensured that all local assessments yield comparable results 
for all subgroups? 

(d) How has the State ensured that all local assessments yield results that can be 
aggregated with those from other local assessments and with any statewide 
assessments? 

(e) How has the State ensured that all local assessments provide unbiased, rational, 
and consistent determinations of the annual progress of schools and LEAs 
within the State? 
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SECTION 3: OVERVIEW OF THE STATEWIDE ASSESSMENT SYSTEM 

(continued) 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






3.3 

If the State’s assessment system employs a matrix design — that is, multiple forms 
within a content area and grade level— how has the State ensured that: 

(a) All forms are aligned with the State’s academic content and achievement 
standards and yield comparable results? 

(b) All forms are equivalent to one another in terms of content coverage, difficulty, 
and quality? 

(c) All assessments yield comparable results for all subgroups? 






3.4 

How has the State ensured that its assessment system will provide coherent 
information for students across grades and subjects? 

(a) Has it indicated the relative contribution of each assessment to ensure alignment 
to the content standards and determining adequate yearly progress? 

(b) Has the State provided a rational and coherent design that identifies the 
assessments to be used? 






3.5 

If its assessment system includes various instruments (e.g., the regular assessment in 
English and either a native-language version or simplified English version of the 
assessment), how does the State demonstrate comparable results and alignment with 
the academic content and achievement standards? 






3.6 

How does the State’s assessment system involve multiple measures, that is, measures 
that assess higher-order thinking skills and understanding of challenging content? 
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SECTIOIS 


[4: TECHNICAL QUALITY 


Can Answer 
Fully/ 
Positively 


We Need 
to do Some 
Work 


Critical Element 






4.1 

For each assessment, including alternate assessment(s), has the State documented the 
issue of validity (in addition to the alignment of the assessment with the content 
standards), as described in the Standards for Educational and Psychological Testing 
(AERA/APA/NCME, 1999), with respect to aU of the following categories: 

(a) Has the State specified the purposes of the assessments, delineating the types 
of uses and decisions most appropriate to each? and 

(b) Has the State ascertained that the assessments, including alternate 
assessments, are measuring the knowledge and skills described in its 
academic content standards and not knowledge, skills, or other characteristics 
that are not specified in the academic content standards or grade level 
expectations? and 

(c) Has the State ascertained that its assessment items are tapping the intended 
cognitive processes and that the items and tasks are at the appropriate grade 
level? and 

(d) Has the State ascertained that the scoring and reporting structures are 
consistent with the sub-domain structures of its academic content standards 
(i.e., are item interrelationships consistent with the framework from which 
the test arises)? and 

(e) Has the State ascertained that test and item scores are related to outside 
variables as intended (e.g., scores are correlated strongly with relevant 
measures of academic achievement and are weakly correlated, if at all, with 
irrelevant characteristics, such as demographics)? and 

(f) Has the State ascertained that the decisions based on the results of its 
assessments are consistent with the purposes for which the assessments were 
designed? and 

Has the State ascertained whether the assessment produces intended and unintended 
consequences? 






4.2 

For each assessment, including alternate assessment(s), has the State considered the 

issue of reliability, as described in the Standards for Educational and Psychological 

Testing (AERA/APA/NCME, 1999), with respect to aU of the following categories: 

(a) Has the State determined the reliability of the scores it reports, based on data for 
its own student population and each reported subpopulation? and 

(b) Has the State quantified and reported within the technical documentation for its 
assessments the conditional standard error of measurement and student 
classification that are consistent at each cut score specified in its academic 
achievement standards? and 

(c) Has the State reported evidence of generalizability for all relevant sources, such 
as variability of groups, internal consistency of item responses, variability 
among schools, consistency from form to form of the test, and inter- rater 
consistency in scoring? 
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SECTION 


f 4: TECHNICAL QUALITY (continued) 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






4.3 

Has the State ensured that its assessment system is fair and accessible to all students, 
including students with disabilities and students with limited English proficiency, 
with respect to each of the following issues: 

(a) Has the State ensured that the assessments provide an appropriate variety of 
accommodations for students with disabilities? and 

(b) Has the State ensured that the assessments provide an appropriate variety of 
linguistic accommodations for students with limited English proficiency? and 

(c) Has the State taken steps to ensure fairness in the development of the 
assessments? and 

Does the use of accommodations and/or alternate assessments yield meaningful 
scores? 






4.4 

When different test forms or formats are used, the State must ensure that the meaning 
and interpretation of results are consistent. 

(a) Has the State taken steps to ensure consistency of test forms over time? 

In the State administers both an online and paper and pencil test, has the State 
documented the comparability of the electronic and paper forms of the test? 






4.5 

Has the State established clear criteria for the administration, scoring, analysis, and 
reporting components of its assessment system, including alternate assessment! s) and 
does the State have a system for monitoring and improving the on-going quality of 
its assessment system? 






4.6 

Has the State evaluated its use of accommodations? 

(a) How has the State ensured that appropriate accommodations are available to 
students with disabilities and that these accommodations are used in a manner 
that is consistent with instructional approaches for each student, as determined 
by a student’s IEP or 504 plan? 

(b) How has the State determined that scores for students with disabilities that are 
based on accommodated administration conditions will allow for valid 
inferences about these students' knowledge and skills and can be combined 
meaningfully with scores from non-accommodated administration conditions? 

(c) How has the State ensured that appropriate accommodations are available to 
limited English proficient students and that these accommodations are used as 
necessary to yield accurate and reliable information about what limited English 
proficient students know and can do? 

(d) How has the State determined that scores for limited English proficiency 
students that are based on accommodated administration circumstances will 
allow for valid inferences about these students' knowledge and skills and can be 
combined meaningfully with scores from non-accommodated administration 
circumstances? 
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SECTION 5: ALIGNMENT OF ACADEMIC CONTENT STANDARDS, 
ACADEMIC ACHIEVEMENT STANDARDS, AND ASSESSMENTS 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






5.1 

Has the State outlined a coherent approach to ensuring alignment between each of its 
assessments, including alternate assessment(s), or combination of assessments, and 
the academic content standards and academic achievement standards the assessment 
is designed to measure? 






5.2 

Are the assessments and the standards aligned comprehensively, meaning that the 
assessments reflect the full range of the State’s academic content standards? Are the 
assessments as cognitively challenging as the standards? Are the assessments and 
standards aligned to measure the depth of the standards? Do the standards reflect the 
degree of cognitive complexity and level of difficulty of the concepts and processes 
described in the standards? 






5.3 

Are the assessments and the standards aligned in terms of both content (knowledge) 
and process (how to do it), as necessary, meaning that the assessments measure what 
the standards state students should both know and be able to do? 






5.4 

Do the assessments reflect the same degree and pattern of emphasis as are 
reflected in the State’s academic content standards? 






5.5 

Do the assessments yield scores that reflect the full range of achievement implied by 
the State’s academic achievement standards? 






5.6 

Assessment results must be expressed in terms of the achievement standards, not just 
scale scores or percentiles. 






5.7 

What ongoing procedures does the State use to maintain and improve alignment 
between the assessment(s) and standards over time? 
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SECTION 6: INCLUSION OF ALL STUDENTS IN THE ASSESSMENT 
SYSTEM 


Can Answer 
Fully/ 
Positively 


We Need 
to do Some 
Work 


Critical Element 






6.1 

Do the State’s participation data indicate that all students in the tested grade levels or 
grade ranges are included in the assessment system (e.g., students with disabilities, 
students with limited English proficiency, economically disadvantaged students, 
race/ethnicity, migrant students, homeless students, etc.)? 






6.2 

1 . What guidelines does the State have in place for including all students with 

disabilities in the regular assessment system? 

2. If the State has approved/adopted alternate achievement standards, what guidelines 

does the State have in place for assessing only students with the most significant 

cognitive disabilities based on alternate achievement standards? 

(a) Has the State developed clear guidelines for Individualized Educational Program 
(IEP) teams to apply in determining when a child’s cognitive disability justifies 
assessment based on alternate academic achievement standards? 

(b) Has the State ensured that parents of those students are informed that their 
child’s achievement will be based on alternate achievement standards? 

(c) Has the State documented that students with the most significant cognitive 
disabilities are, to the extent possible, included in the general curriculum and 
assessments aligned to that curriculum? 

(d) Has the State developed, disseminated information on, and promoted use of 
appropriate accommodations to increase the number of students with the most 
significant cognitive disabilities who are tested against grade-level academic 
achievement standards? 

(e) Has the State ensured that regular and special education teachers and other 
appropriate staff know how to administer assessments, including making use of 
accommodations, for students with the most significant cognitive disabilities? 






6.3 

What guidelines does the State have in place for including all students with limited 

English proficiency in the tested grades in the assessment system? 

(a) Has the State made available assessments, to the extent practicable, in the 
language and form most likely to yield accurate and reliable information on 
what these students know and can do? 

(b) Does the State require the participation of every limited English proficient 
student in the assessment system, regardless of how long a student has been 
enrolled in US schools? Has the State adopted policies requiring limited English 
proficient students to be assessed on the reading/language arts standards in 
English if they have been enrolled in US schools for three consecutive years or 
more? 






6.4 

What policies and practices does the State have in place to ensure the identification 
and inclusion of migrant and other mobile students in the tested grades in the 
assessment system? 
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SECTION 7: AN EFFECTIVE SYSTEM OF ASSESSMENT REPORTS 


Can 

Answer 

Fully/ 

Positively 


We Need 
to do 
Some 
Work 


Critical Element 






7.1 

Does the State’s repotting system facilitate appropriate, credible, and defensible 
interpretation and use of its assessment data? 






7.2 

Does the State report participation and assessment results for all students and for 
each of the required subgroups in its reports at the school, LEA, and State levels? In 
these assessment reports, how has the State ensured that assessment results are not 
reported for any group or subgroup when these results would reveal personally 
identifiable information about an individual student? 






7.3 

How has the State provided for the production of individual interpretive, descriptive, 

and diagnostic reports following each administration of its assessments? 

(a) Do these individual student reports provide valid and reliable information 
regarding achievement on the assessments in relation to the State’s academic 
content and achievement standards? 

(b) Do these individual student reports provide information for parents, teachers, 
and principals to help them understand and address a student's specific academic 
needs? Is this information displayed in a format and language that is 
understandable to parents, teachers, and principals and are the reports 
accompanied by interpretive guidance for these audiences? 

(c) How has the State ensured that these individual student reports will be delivered 
to parents, teachers, and principals as soon as possible after the assessment is 
administered? 






7.4 

How has the State ensured that student-level assessment data are maintained securely 
to protect student confidentiality? 






7.5 

How has the State provided for the production of itemized score analyses so that 
parents, teachers, and principals can interpret and address the specific academic 
needs of students? 
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